1. Field of the Invention
The present invention relates generally to an image processing and modulation system for a digital display system, and more particularly to performing field sequential lighting control along with advanced image processing and image modulation to produce high-quality output images.
2. Discussion of Prior Art
The many types of Flat Panel Display (FPD) technologies include Active Matrix Liquid Crystal Displays (AMLCDs) also referred to as Thin Film Transistor displays (TFTs), silicon Reflective LCDs (si-RLCDs), Liquid Crystal On Silicon (LCOS), ferroelectric displays (FLCs), Field Emission Displays (FEDs), Carbon Nanotube based Nano Emissive Displays (NEDs), ElectroLuminescent Displays (ELDs), Light Emitting Diodes (LEDs), Organic LEDs (OLEDs), Plasma Displays (PDs), Passive matrix Liquid Crystal Displays (LCDs), Thin Film Transistor (TFT), Silicon X-tal Reflective Displays (SXRD) and Digital Mirror Displays (DMDs).
Manufacturing FPDs requires special processing steps and it is often difficult to achieve acceptable visual quality in terms of consistence across the entire area of the display. Besides being a yield issue for new displays, display characteristics can change over operating conditions and over the lifetime of the display. Most FPD technologies also require light sources, such as fluorescent lamps, Cold Cathode Florescent lamps, Ultra High Power (UHP) lamps or Light Emitting Diodes (LEDs) to illuminate the display images. LEDs may be made from a variety of materials and each may have their own “color temperature” characteristics. Other systems may use a hybrid of a traditional lamp along with one or more white or colored LED arrays. In one example system, Sharp Corporation of Japan uses a cold cathode florescent lamp along with red LEDs in a hybrid back light approach for 37″ and 57″ LCD TVs.
Some of these technologies can be viewed directly as FPDs and others are used as microdisplay devices where the image is projected and a user views the projected image.
Projection display systems may use transmissive or reflective “microdisplay” technologies. To achieve the desired full color gamut in microdisplay based parallel color projection systems, three separate microdisplay image modulators, one for each of the R, G, and B (RGB) components, may be used. A single microdisplay image modulator which produces R, G, and B, either through spatial color filters or with sequential color fields at a sufficiently high rate can cost less. Other light sources such as LEDs and lasers may also be used for the different colors with one, three or more imagers. Reflective technologies such as Texas Instruments Digital Light Processing (TI DLP) and technologies such as LCOS imagers are other examples of popular microdisplay technologies. Sony uses a Silicon X-tal Reflective Display (SXRD) microdisplay in some of their rear projection television systems. Other display devices may be based on High Temperature Poly-Silicon (HTPS) technology, Micro Electro-Mechanical Systems (MEMS) or Carbon Nanotubes.
Projection displays have a projection path with associated additional properties. For example, projecting an image from a projector on a level below the middle of a flat screen results in an image which is narrower at the bottom than at the top in what is known as the “Keystone” effect. Radial distortion occurs when an image pixel is displaced from its ideal position along a radial axis of the image. Because an image has the largest field angles in the display corners, the corners exhibit worse radial distortion than other display areas. Radial distortion includes barrel distortion, where image magnification decreases towards the corners, and pin cushion distortion, where the magnification increases towards the corners. Lens related distortions including radial distortion can cause image deformation. Distortion can also result from non-flat screens or the Earth's magnetic field. Shortening the projection path, often done to thin a rear projection system, also increases distortion. The non-uniform projection characteristics of a display system may be represented by a “distortion map” which is based on a single pixel or a group of pixels.
Types of three dimensional binocular display systems include anaglyph displays, frame sequence displays, autostereoscopic displays, single and multi-turn helix displays. These normally have multiple camera data channels. Anaglyph systems usually require a user to wear red and green (or red and blue or red and cyan) glasses so that each eye perceives a different view. Frame sequencing systems use shutter glasses to separate left and right views. Autostereoscopic displays use lenticular lenses and holographic optical elements. Single or multi-turn helix displays use multiple semi-transparent display screens which can be seen by multiple observers without special glasses.
There is a finite time required to load an image into an array so that the entire frame is available for display. A TFT display typically includes column and row drivers that are used to capture a serial frame data stream and scan the values into the display array. Some displays illuminate the image array during the loading process and other displays wait until the entire image is loaded and “flash” the illumination. Because of inherent response characteristics of LCD materials and the control of the illumination, LCD-based projection and LCD-based direct view display systems each have unique flicker characteristics and exhibit different motion artifacts. Further, LCDs switch in the finite time it takes to change the state of a pixel. Active matrix thin film transistor (TFT) displays, which have an active transistor controlling each display pixel, require a switching time related to the LCD material composition and thickness, and to the techniques of switching. The transitions from one state to another are also not necessarily linear and can vary depending on the sequence of pixel values.
The ability to accurately portray each color component for each pixel at each screen location is another desired function of a high quality display device. Since a display output pixel typically consists of a triad (red, green, blue) or other combination of color components, each color component for each pixel is typically made up of multiple sub-pixels. However, the term “pixel” is also used to describe the triad of sub-pixels. Each color component of the pixels mapped to the screen display pixels may transfer non-uniformly where some pixels end up too bright and others end up not bright enough. This non-uniformity may be a function of the light path characteristics, the mirrors, the lenses, the lamps or any combination thereof. Similarly for LED backlit FPDs, the spacing and positioning of the LEDs may cause non-uniform brightness patterns where the image modulator pixels closer to the LEDs appear brighter than the image modulator pixels further from the LEDs. In the case of colored LEDs or LEDs with color filters, the modulator pixels may be further affected not just with brightness differences, but with color uniformity differences. The brightness of LEDs can be modulated using various Pulse Width Modulation (PWM) techniques or varied by adjusting the voltage or current. The LEDs may be controlled collectively, by proximity in groups, by color, by some combination or in the most sophisticated case each LED can be controlled individually.
FIG. 1A shows a front view of an LED array 102f made up of a structure of multiple LEDs 104. Each LED is a light source which can be either a white LED or one of a variety of colors such as red, green or blue. Other LED colors are also used and the color may be part of the LED structure, may be a color filter as part of each LED or may be a filter mechanism for the array of LEDs. The array of LEDs may be arranged in a pattern of colors so as to produce a controllable and consistent light source that, when combined with an image modulator, is capable of producing the full gamut of display colors.
FIG. 1B shows a side view of a light source subsystem 100 used in a projection system. Subsystem 100 includes an LED array 102s structured from multiple LEDs 104, a multi-lens system 108 and 112 and a microdisplay imager or image modulator 110 that may use a variety of light modulation techniques. The LEDs may be white or multicolored and an additional color system (not shown), such as a color wheel, may also be included in system 100. In one example system, lens 108 concentrates the light through the microdisplay 110, and lens 112 controls the light projection. The projection system may be front or rear projection and use other lenses and mirrors, dichroic recombiners and filters to project the image on a screen (not shown). Microdisplay 110 may be either transmissive, passing modulated amounts of light through each pixel, or reflective, reflecting modulated amounts. The resolution of the microdisplay 110 is typically much higher than the resolution of the components that make up the light source. Projection systems utilizing a color wheel, such as a TI DLP based system, may sequentially project Red, Green and Blue color fields of the image. Other projection systems may use separate image modulators 110 per color component.
FIG. 1C shows a side view of a direct view LCD panel subsystem 140 where a backlight 122 is made up of either strips 122 or an array 102 of LEDs. The LCD display or image modulator 130 typically has a much higher resolution than the resolution of the components that make up the light source. Exceptions include OLEDs, where each LCD pixel has a light source as part of the pixel and as such the resolution of the light sources is the same as that of the display and NEDs, where carbon nanotubes are arranged such that a properly applied voltage excites each nanotube which in turn bombards the color phosphors for the pixels or subpixels with electrons and “lights them up.” The brightness and color wavelength is based on the quantity of electrons, the phosphors used and a variety of other factors.
For LED backlit displays, various filters, color and light diffusion gradients, Brightness Enhancement Films (BEF), diffusion plates, light guides and mixing light guides 126 may be used as an option to improve the light uniformity for the display. Some backlit displays may also include one or more optical sensors (not shown in FIG. 1C) that can detect changes in brightness and wavelength (see Hamamatsu product brief). A typical direct view system includes a color filter 132 that is closely matched to the sub pixels of the image modulator. Depending on the materials and fabrication steps, the color filter may be a pattern of R, G and B stripes or other block arrangements similar to Bayer patterns that are “overlaid” or “printed” onto an image modulator's screen. The image modulator may include three or four times the sub-pixel sites so that there is separate control for each R, G and B sub-pixel component of each pixel. Since color filters typically operate by filtering out wavelengths from a white light source, some lumen efficiency is lost by using color filters.
Most image modulators (110 and 130) are addressed in raster scan fashion and each pixel is refreshed during each display frame interval. Row and Column drivers are typically used in the scanning process and every output pixel is written to the display during every refresh cycle regardless of whether the value of the pixel has changed since the last cycle. Each R, G, and B color component normally has a different intensity value which is digitally represented by a number of bits. For example, if 8 bits represent each R, G, and B color component, then each component has 28 (=256) intensity values from 0 to 255. Changing the intensity value of a color component in an ideal digital device from a number X, for example, to a number Y, takes just as long regardless of the Y value. So in an ideal system, changing a color component value from 2 to 3 takes as long as changing the value from 2 to 200. However, because of the nature of LCD image modulator pixels, the transitions for modulating light intensities are not purely digital, and are not necessarily linear. U.S. Pat. No. 6,340,994 by Margulis et al. teaches Temporal Gamma Processing (TGP) which assures that the time-related representations of an image are as accurate as possible, based on a previous frame value and a known transfer function of the display modulation system and adjusting its output to a desired value during display of a desired frame.
The light sources for a modulator based system may not be able to achieve the desired brightness or accurately and uniformly reproduce the intended color gamut. Different modulator pixel positions on the screen may be affected by the light sources differently, causing a display of non-uniform color and brightness. Additionally, the light sources' output characteristics may change over time and need to be compensated for.
Therefore, for all the foregoing reasons, what is needed is an image processing system to effectively enhance display quality by controlling both the image modulator and controlling colored light sources and thereby provide better visual images.